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Abstract — In this letter we consider the ensemble of codes 
formed by the serial concatenation of a Hamming code and two 
accumulate codes. We show that this ensemble is asymptotically 
good, in the sense that most codes in the ensemble have 
minimum distance growing linearly with the block length. Thus, 
the resulting codes achieve high minimum distances with high 
probability, about half or more of the minimum distance of 
a typical random linear code of the same rate and length in 
our examples. The proposed codes also show reasonably good 
iterative convergence thresholds, which makes them attractive 
for applications requiring high code rates and low error rates, 
such as optical communications and magnetic recording. 



I. Introduction 

Applications such as magnetic recording or fiber-optic com- 
munications require error-correcting codes with a very high 
code rate (R> 0.8) and simple decoding algorithms amenable 
to high-throughput decoding architectures. Following the in- 
vention of turbo codes [1] and the rediscovery of Gallager's 
low-density parity-check (LDPC) codes [2], several high- 
rate low-complexity capacity-approaching codes have been 
proposed in the literature. Examples of such codes include 
turbo product codes (TPC) [3], the serial concatenation of 
a Hamming code with an accumulate code [4] or structured 
LDPC codes (see e.g. [5, chap. 17]). While such codes usually 
offer very good performance in the waterfall region, they 
are not asymptotically good in the sense that, unlike random 
codes, their minimum distance dmin does not grow linearly 
with block length, and thus may not be large enough to achieve 
the required error rates (e.g. of the order of 10~^^ or lower 
for optical communications). 

Recently, it was shown that repeat multiple- accumulate 
(RMA) codes with two or more accumulate stages are 
asymptotically good [6,7]. It was further shown in [6] that 
high-rate code ensembles obtained by puncturing a low-rate 
repeat-accumulate-accumulate (RAA) code yield linear dis- 
tance growth close to the Gilbert- Varshamov Bound (GVB). 
Unfortunately, as we shall see in Section IV, iterative decoding 
of punctured RMA does not converge for very high rates, 
making them impractical. In order to overcome this limitation, 
we consider in this work a class of high-rate double serially 
concatenated codes based on an outer, possibly extended 
Hamming code, with two accumulate codes. We first study 
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Fig. 1. Serial concatenation of an (n, k) block code Co and two accumulators 
Cl and C2. 



the ensemble-average finite-length weight enumerator for this 
code ensemble. Then, by generalizing the analytical tools 
introduced in [6,7] to handle the case of an arbitrary outer 
linear block code, we study the asymptotic growth rate of 
the weight enumerator. We show, through selected examples, 
that the typical minimum distance of Hamming-accumulate- 
accumulate (HAA) codes grows linearly with block length, and 
provide a numerical estimate of the growth rate. Finally, we 
use extrinsic information transfer (EXIT) charts to estimate the 
iterative convergence thresholds. It is shown that the proposed 
codes have reasonably good convergence thresholds despite 
the double serial concatenation. Compared to TPCs much 
larger minimum distances can be achieved at the expense of 
a moderate loss in code convergence. Thus, HAA codes are a 
valid alternative when very high code rates and very low error 
rates are sought for. 

II. Encoder structure and finite-length ensemble 

ENUMERATORS 

We consider the code ensemble C formed by the serial 
concatenation of an (n, /c) outer block code Cbc and two rate- 
1, memory-one, accumulate codes Ci and C2 with generator 
polynomials g{D) = 1/(1 + 1)), connected through two 
interleavers tti and 7r2. We assume that the interleaver size 
TV is a multiple L of the length n of the outer block code. 
The overall code rate is R = K/N , where K = kL is the 
input block length and N = nL is the output block length. 
We denote by Co the (N^K) outer block code obtained by 
concatenating together L successive codewords of Cbc • Trellis 
termination is used to transform the accumulate codes Ci and 
C2 into two equivalent (Ni^Ki) block codes, / = 1,2. The 
corresponding encoder is depicted in Fig. 1. 

In this letter, we consider Hamming and extended Hamming 
(eHamming) codes for Cbc since they can achieve minimum 
distances 3 and 4 with the highest code rate for a given 
dimension, with reasonable decoding complexity. A uniform 
distribution is assumed on the choice of the two permutations 
TTl and 7T2. In the following, the code ensemble C with an outer 
code of parameters (n, /c) will be referred to as the (n, k)AA 
ensemble. 

Let ^ denote the input-output weight enumerator 
(lOWE) of a block code C, i.e. the number of codewords 
with input weight w and output weight h in C. Similarly, let 
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= ^ denote the weight enumerator (WE) of the 

code, i.e. the number of codewords of output weight h. Then, 
using the uniform interleaver concept [8,9], the ensemble- 
average WE of the code ensemble C can be computed as: 
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The lOWE of an accumulate code with block length N can 
be written in closed form as [10]: 

^A^ - /i\ / h-1 
Xw/2\)[\w/2]-l 

In polynomial form, the WE of Co and Cbc are linked by 
the following relationship [8]: 
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Here, we derive another alternative expression of the WE of 
Co which is more convenient for asymptotic analysis. Consider 
a codeword c of output weight ho in Co and denote by the 
number of codewords of weight i in Cbc that participate in c. 
Then, the WE of the outer block code, , can be expressed 
as 
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under the constraints XlILo ' 
and where 
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is the multinomial 
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coefficient. 

The ensemble-average WE can be used to bound the mini- 
mum distance dmin of the code ensemble C. In particular, the 
following Proposition holds [11]: 

Proposition 1: The probability that a code randomly chosen 

c 

from an ensemble of linear codes C with average WE has 
dmin < (i is upper bounded by 



d-i 



Pr(c^min <d)<^Ay^ 



(5) 
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In Fig. 2 we display this probabilistic bound for four 
(n, A:)AA code ensembles by plotting the largest weight d in 
the right-hand side (RHS) of (5) yielding YllX'^h < 1/2, 
as a function of the code length A^. Hence we expect at 
least half of the codes in C to have a minimum distance 
dmin at least equal to the value predicted by the curves. The 
considered outer codes are the (31, 26) and (63, 57) Hamming 
codes, as well as the (32, 26) and (64, 57) eHamming codes. 
A typical random linear code of length A^ and rate R has 
minimum distance A^^gv [12], where (5gv is the normalized 
Gilbert- Varshamov distance defined as the root 5 < 1/2 of 
the equation El2(^) = 1 — R, and El2(x) is the binary entropy 
function (with binary logarithm). For comparison purposes, we 
have also plotted in Fig. 2 the minimum distance predicted by 
the GVB for rates R = 26/32, R = 26/31 and R = 57/63 
(the GVB for = 57/64 is omitted for clarity). All codes 
appear to have a minimum distance that grows linearly with the 
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Fig. 2. Probabilistic bound on the minimum distance dm in versus block 
length N for several (n, k)AA code ensembles. 

block length. Furthermore, the achievable minimum distances 
are very high. For instance, the bound for the (31,26)AA 
ensemble predicts a typical minimum distance dmin ^ 117 
for block length A^ = 8184. For comparison, the product 
code (128, 120) x (64, 57) of similar code length and rate has 
dmin = 16 only. 

III. Asymptotic ensemble weight enumerator 

ANALYSIS 

In this Section we analyze the asymptotic behavior of the 
WE to show that the minimum distance of the considered 
(n, k)AA code grows linearly with code length. To this end, 
we study the behavior of the spectral shape function of the 
code ensemble C, defined as [2] 

1 



where S = h/N = h/{nL) is the normalized output 
weight. From (6) the WE can be expressed as A^ ~ QNr{6)^ 
Therefore, if there exists some abscissa Smin > such that 
sup^<^r(x) < < Jmin, and r{5) > for some 
S > ^min, then it can be shown (using Proposition 1 for 
example) that, with high probability, the minimum distance 
of most codes in the ensemble grows linearly with the block 
length A", with growth rate dmin [7, 13]. On the other hand, 
if r{S) is equal to zero rather than strictly negative in the 
interval (0, Smin), it cannot be concluded directly whether the 
minimum distance grows linearly or not with A^ since the RHS 
in (5) may be bounded away from zero. As shown recently in 
[7], the spectral shape of RMA code ensembles exhibits such 
a behavior. However, based on WE analysis and appropriate 
bounding techniques, the authors of [7] were able to prove 
that for such code ensembles, the typical minimum distance 
indeed grows linearly with A", with growth rate ^min- 

A. Spectral shape of the proposed code ensemble 

Consider the {Ni, Ki) block code Cu ^ = 0, 1, 2. We define 
the asymptotic (logarithmic) behavior of the WE for Ci as the 
function 
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where f3i = hi/Ni is the normahzed output weight. Similarly, 
we define the asymptotic behavior of the lOWE for Ci as the 
function 

a^Hai.Bi) = lim sup-^lnAp^ \ \ r at \ (8) 

where ai = wi/ Ki is the normalized input weight. Then, using 
(1) and (7-8) in (6) and recalling Stirling's approximation for 
binomial coefficients (^) ^''^eik/n) ^^^^^ jj^^.) 

binary entropy function with natural logarithms, the spectral 
shape function of the code ensemble C can be written as 

N N 

r{5) = Jim^sup-ln ^ ^ exp{Ara^°(/3o) 

/io=0 /ii=0 

+ iVa^n/3o,/3i)+A^a^=(/?i,/3) 
-ArHe(/3o)-AfHe(/3i)} 

{a^''(/3o)+a^^(/3o,/3i) 

0<Po,Pi<l 

+a^^(A,/5)-He(/5o)-He(A)} 

where /^o = /?^o/^ and Pi = hi/N. The last line follows 
from the well-known max-log approximation ln(e" + e^) 
max(a, 6) (see [14]). 

The asymptotic behavior of the lOWE of the accumulate 
code Ci, I = 1,2, is easily obtained by invoking again 
Stirling's approximation in (2), yielding [10], 

«-.(»-,A) = (i-fl)M.(^)+»(i|) 

(10) 

The next Proposition addresses the problem of computing 
the asymptotic weight enumerator a^^{(3o) for the outer block 
code Cq. 

Proposition 2: Let Co be the {N^ K) block code obtained 
by concatenating together L successive codewords of an 
{n,k) block code Cbc- Let pi be the relative proportion of 
codewords of Cbc of weight i in a codeword of Co, i.e. 
Pi = rrii/L. Define P = (po,Pi, • • • ,Pn) and H(P) = 
— J27=oP'i^^P'i^ with the convention OlnO = 0. Then, the 
asymptotic lOWE of Co is given by the solution of the 
following convex optimization problem 

a^°(/5o) = max^ ( H(P) + f^p, In Af^M (11) 

under the constraints X^ILo ~ ^/^o and X^ILo^* ~ ^• 

Proof: The proof follows the approach proposed in [14] 
to obtain the spectral shape of generalized LDPC codes 
employing small Hamming codes at the check nodes. Since 
our problem is simpler, however, we arrive at a more tractable 
optimization problem (only the knowledge of the WE of Cbc 
is required) and we avoid the conjectures made in [14]. 

Recalling first that Nq = N = nL, we can express a^°(/5o) 
in (7) as a function of L, 

{(3o) = lirn^ sup In A^^^ (12) 

with Po = ho /No = ho/nL. Now, define the type P = 
{Po^Pi: • • • :Pn) of a codeword c in Co, c = c^c^...c^. 



with G Cbc. as the relative proportion of occurrences of 
codewords of Cbc of weight i, i = 0, 1, . . . , n, in c [15]. The 
set of all length-L sequences each containing rrii occurrences 
of codewords of Cbc with weight i is called the type class 
of P, denoted r(P). It follows that |r(P)| = {^^^^f^^^^J- 
From [15, Thm. 11.1.3] we have that 

|T(P)| ^-^e^^(P) (13) 

Finally, using (13) in (4) and (12) we obtain: 

= lim sup ^ In V QLU{p)(^^CBcyoL ,,,(^^CBcynL 

(14) 

from which (11) follows. ■ 
The asymptotic WE (11) admits a closed-form expression 

for a few simple codes such as the {nK, K) block repetition 

code of rate R= 1/n. 

Example 1: Asymptotic WE of the {nK^K) repetition code 
Let Co be the {nK, K) block code formed by concatenating 

together K codewords of an (n, 1) repetition code Cbc- We 

have ^0 = A^^^ = 1 and = for i = 1, . . . , n - 1. 

Define po = mo/K and pn = rrin/K. Using (11) we obtain: 

a^^{po) = max - ( H(P) + f^Pz In ^f^^ ) 

^ ^ V i=o J (15) 

max - {-po \npo Pn Inpn) 

po.Pn n 

under the two constraints npn = nPo and po -\- Pn = 1- 
Therefore, 

a^n/^o) = - (-(1 - Po) ln(l - Po) - Po ln/?o) = -^e{Po) 

" (16) 

which is a well-known result (see e.g. [6, 10]). 

For more general block codes with known WE, (Po) can 
be evaluated numerically using standard convex optimization 
software. 

B. Asymptotic growth rate of the minimum distance 

The numerical evaluation of (9) for the (31,26)AA, 
(32,26)AA, (63,57)AA and (64, 57)AA codes ensembles is 
shown in Fig. 3. We have also plotted the spectral shape 
r{S) = Me{S) — {1 — R) ln(2) for the corresponding random 
linear code ensembles. The behavior of the spectral shape of 
the (n, A:) AA ensembles is similar to the one obtained in [7] for 
RMA codes. It is strictly positive in the range (^min, 1 — ^min) 
for some < ^min < 1/2, and zero elsewhere. Since a closed 
form expression of the WE of the outer block code Co is not 
available, in contrast to RMA codes [7] we cannot provide a 
formal proof that the considered code ensembles are asymptot- 
ically good. However, extensive numerical experiments using 
(5) show that Pr((imin < L^minA/'J) — > as A/" gets large, 
which suggests that the results of [7] hold true for more general 
outer codes than just repetition codes. In Table I we report the 
estimated values of Smin for several (n, k)AA ensembles based 
on high-rate Hamming and extended Hamming codes. For 
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Fig. 3. Spectral shape of selected (n, k)AA code ensembles. The spectral 
shape of the corresponding random linear codes of same rate is also shown 
in dashed lines. 

TABLE I 



Asymptotic normalized minimum distance S^i^ and iterative 

CONVERGENCE THRESHOLD FOR SELECTED (n, k)AA CODE ENSEMBLES. 



Code 


^min 




Threshold 


Constrained capacity 


(32, 26)AA 


0.0197 


0.0286 


3.34 dB 


2.14 dB 


(31, 26) AA 


0.0140 


0.0236 


3.48 dB 


2.39 dB 


(64, 57)AA 


0.0091 


0.0145 


4.10 dB 


3.03 dB 


(63, 57)AA 


0.0067 


0.0122 


4.20 dB 


3.26 dB 


(128, 120)AA 


0.0042 


0.0073 


4.70 dB 


3.93 dB 


(127, 120)AA 


0.0032 


0.0063 


4.79 dB 


4.11 dB 



comparison purposes, the normalized minimum distance Sqv 
of random linear codes is also given. The estimated asymptotic 
growth rates Smin are in good agreement with the slope of 
the curves obtained by the finite-length analysis in Section II. 
Hence we conclude that HAA codes have minimum distance 
growing linearly with block length as ^min^- 

IV. Convergence analysis 

In this section, we investigate the iterative convergence 
behavior of (n, k)AA code ensembles by means of an EXIT 
chart analysis [16]. The iterative convergence thresholds for 
the considered (n, A:)AA codes ensembles are given in Table I. 
BPSK transmission over an AWGN channel is assumed. 
Convergence is achieved within 0.7 dB-1.2 dB from the 
corresponding Shannon limit. Furthermore, the thresholds get 
closer to capacity for higher rates. For instance, the predicted 
thresholds for the (31,26)AA and the (63, 57)AA code en- 
sembles are 3.48 dB and 4.20 dB, respectively, i.e. 1.09 dB 
and 0.94 dB away from the constrained capacity. We also 
note that Hamming outer codes may be preferable to extended 
Hamming outer codes in practice since the former provide 
better thresholds at higher code rate and with smaller decoding 
complexity (number of states reduced by half in the code 
trellis), at the expense of a slightly smaller asymptotic growth 
rate of the minimum distance. 

Comparison of the proposed codes with Hamming- 
accumulate codes, denoted hereafter as (n, /c)A codes, show 
that the latter have better convergence thresholds. For example, 
the (31,26)A and (63, 57)A codes converge at 2.81 dB and 
3.58 dB, respectively. Thus double concatenation incurs a loss 
of 0.6 dB with respect to single concatenation. On the other 



TABLE II 

Normalized minimum distance 6min for selected 

RANDOMLY-PUNCTURED R3 AA CODES WITH TARGET RATE R. 



R 


<5min 




26/32 


0.0282 


0.0286 


26/31 


0.0233 


0.0236 


57/64 


0.0143 


0.0145 


57/63 


0.0120 


0.0122 


120/128 


0.0072 


0.0073 


120/127 


0.0062 


0.0063 



hand, Hamming-accumulate codes (and more generally BCH- 
accumulate codes [17, 18]) are asymptotically bad. 

It was shown in [6] that the ensemble of randomly- 
punctured RAA codes is asymptotically good and achieves 
linear minimum distance growth close to the GVB, even 
for very high rates. As an example, we have reported in 
Table II the normalized minimum distance Smin for punctured 
RAA codes with repetition factor 3 (hereafter denoted as 
R3AA) and same rate than the (n, A:)AA codes considered 
in Table I. The results show that punctured R3AA codes 
significantly outperform HAA codes in terms of normalized 
minimum distance. On the other hand, as noted in [6,11], 
the convergence thresholds of RAA under iterative decoding 
are generally away from capacity. The situation gets even 
worse in the presence of puncturing. Our experiments have 
shown the existence of a maximum rate i?max above which 
the EXIT chart for randomly-punctured RAA codes shows an 
early cross between the two EXIT curves, even at very high 
(ultimately infinite) Ei^/Nq, meaning that iterative decoding 
cannot converge. This limiting rate is around i^max ^ 0.695 
for punctured R3AA codes. Therefore iterative decoding of 
randomly-punctured R3AA codes do not converge at the code 
rates considered in Tables I and II. Simulations confirmed this 
prediction. Note that lowering the repetition factor does not 
help. For example, we found that i?max ^ 0.42 for ReAA. 

In [19] a family of asymptotically good rate-compatible 
protograph-based LDPC codes was introduced which supports 
any code rate of the form R = (n + l)/(n + 2) for 
n = 0,1,2,... These codes, nicknamed AR4JA, combine 
rate flexibility with excellent convergence thresholds, within 
0.45 dB or less from the constrained capacity at all rates. For 
instance, AR4JA codes converge at 0.37 dB from capacity at 
rate R = 5/6 [19]. This is 0.75 dB better than (31,26)AA 
codes. On the other hand, the minimum distance growth rate 
for AR4JA codes is only Smin = 0.015 for the lowest code 
rate R = 1/2 [19], whereas HAA codes achieve almost the 
same growth rate ((^min = 0.014) but at much higher code rate 

= 26/31 ^ 5/6. Thus, HAA codes are expected to achieve 
significantly higher minimum distance than AR4JA codes of 
same rate and code length. 

V. Simulation results 

We compared the performance of the proposed (n, k)AA 
codes with that of TPCs, structured LDPC codes and (n, k)A 
codes, assuming BPSK modulation and transmission over an 
AWGN channel. In all simulations, random interleavers and 
a maximum of 30 decoding iterations were considered for 
(n, A:)AA codes and (n, k)A codes. Turbo decoding of the 
(n, k)AA codes was realized as described in [9]. An optimum 
trellis-based MAP decoder was used to decode the Hamming 
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Fig. 4. Bit error probability performance for (31, 26) AA and (127, 120) AA 
ensembles on an AWGN channel with BPSK modulation. 

codes. Turbo decoding of the TPC was realized using the 
Chase-Pyndiah decoding algorithm [3] with 16 test patterns 
and a maximum of 16 iterations (no significant improvement 
was observed beyond). One iteration comprises here a row- 
decoding step followed by a column-decoding step. Note that 
the Chase-Pyndiah decoder may also be applied to decode the 
outer block code in the (n, /c)A and (n, /c)AA concatenated 
schemes, resulting in lower complexity with performance very 
close to MAP decoding [18]. 

In Fig. 4 the bit error rate (BER) performance of the 
(31,26)AA code is compared with the performance of the 
(128,120) X (64,57) TPC and (31,26)A code, respectively. 
The three codes have similar rate. The block length is = 
8184 for the (31, 26)AA and (31, 26)A codes and = 8192 
for the TPC. The simulated curve for the (31, 26) AA code is 
in agreement with the convergence threshold predicted by the 
EXIT charts. A loss of ^ 0.6 dB and of ^ 0.5 dB are observed 
with respect to the (31, 26)A code and the TPC, respectively. 
However the (31, 26) AA code is expected to yield significantly 
lower error floor thanks to a much higher (imin- For instance, 
the minimum distance of the TPC is 16 while the majority of 
the codes in the (31,26)AA ensemble are expected to have 
minimum distance ^114 with high probability. 

The performance of the (127, 120)AA code is also given in 
Fig. 4. It is compared with the performance of the (127, 120) A 
code and also with a (10429,9852), rate-i^ = 0.945, graph- 
theoretic LDPC code described in [5, Fig 17.43]. The latter 
was selected because of its code rate R ^ 120/127 and its 
good performance under iterative decoding for a structured 
LDPC code (its Tanner graph is free of length-4 cycles). 
Accordingly, the block length was set to N = 10414 bits 
for both the (127, 120)AA and the (127, 120)A codes. The 
(127, 120)A code performs the best in the waterfall region. 
However, it shows the highest error floor due to a poor 
minimum distance (the floor would be lowered about 2 decades 
by using a carefully optimized interleaver). The (127, 120)AA 
code shows a loss of ~ 0.5 dB and ~ 0.25 dB in the 
waterfall region with respect to the (127, 120) A code and the 
structured LDPC code, respectively. The minimum distance 
of the structured LDPC code is not known precisely but is 



guaranteed to be at least 7 [5, p. 934]. On the other hand, 
at least half of the codes in the (127, 120)AA code ensemble 
have minimum distance 33. The latter are therefore expected 
to perform better at very low error rates. 

VI. Conclusions 

We studied the serial concatenation of a Hamming code 
with two accumulate codes and showed that the resulting code 
ensemble is asymptotically good in the sense that most codes 
in the ensemble have minimum distance growing linearly with 
block size. We described a computational method to estimate 
the asymptotic growth rate of the minimum distance. Although 
only Hamming codes were considered here for the purpose 
of designing high-rate codes, the proposed method naturally 
extends to other outer linear block code. Finally, an EXIT 
chart analysis showed that Hamming-accumulate-accumulate 
codes exhibit reasonably good iterative convergence thresholds 
in spite of the double serial concatenation. 
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